Wide-angle stereoscopic vision with cameras having different parameters

ABSTRACT

A stereoscopic vision system uses at least two cameras having different parameters to image a scene and create stereoscopic views. The different parameters of the two cameras can be intrinsic or extrinsic, including, for example, the distortion profile of the lens in the cameras, the field of view of the lens, the orientation of the cameras, the positions of the cameras, the color spectrum of the cameras, the frame rate of the cameras, the exposure time of the cameras, the gain of the cameras, the aperture size of the lenses, or the like. An image processing apparatus is then used to process the images from the at least two different cameras to provide optimal stereoscopic vision to a display.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 15/903,872, filed Feb. 23, 2018, entitled “Wide-AngleStereoscopic Vision With Cameras Having Different Parameters,” currentlypending, which claims the benefit of U.S. Provisional Patent ApplicationNo. 62/463,350, filed on Feb. 24, 2017, entitled “Wide-anglestereoscopic vision with cameras having different parameter,” the entirecontents of all of which are incorporated by reference herein.

BACKGROUND OF THE INVENTION

Embodiments of the present invention relate to an optical apparatus tocapture multiples images of a wide-angle scene with multiples camerashaving different imaging parameters. In existing systems, to createstereoscopic vision for a human observer, multiple identical camerashaving identical lenses are used to capture the scene from severalviewpoints and simulate the parallax view created by the distance fromthe human eyes. However, this perfect symmetry of both eyes is notrepresentative of real human eyes, where one eye often has differentimaging capabilities or defects than the other and one eye is moreimportant because it has ocular dominance over the other when observinga scene. The present invention uses a combination of hardware cameraswith different imaging parameters combined with software processing tooptimally use the information from the multiples cameras with differentparameters and present the optimal views to the user.

Existing stereoscopic vision system use two or more identical camerasgenerally having lenses with narrow angle FoV to image the scene andcreate stereographic views for an observer. There are some advantages touse identical cameras to observe in stereoscopy the scene, includingdirect compatibilities with display devices without further imageprocessing. However, by using identical cameras, a lot of information iscaptured in double by the cameras just to create the geometricaldifference in the images due to parallax. More useful information couldbe captured if different cameras instead of identical cameras were usedin combination to image processing.

Some existing stereoscopic imaging system use identical wide-anglelenses to observe the scene and allow capturing more field of view thanwhat is viewed by a user at a specific time, allowing the user to modifythe display area inside the full field of view of the wide-angle lenses.However, even if these lenses have a good parallax vision based on theirseparation when looking in a central direction, these wide-angle lensesloose 3D vision when looking in the direction of the axis between thecameras because no more parallax information is present.

In existing stereoscopic vision systems, there are various challenges tooffer a comfortable vision to a human observer considering that theideal display parameters vary from one human observer to the other. Thediscomfort to users can be removed by further image processing to bettercalibrate the two displayed images to the user and mimic perfectly thehuman vision.

BRIEF SUMMARY OF THE INVENTION

To overcome all the previously mentioned issues, embodiments of thepresent invention use at least two cameras having different parametersto image the scene and create stereoscopic views. The differentparameters of the two cameras can be intrinsic or extrinsic, including,but in no way limited to, the distortion profile of the lens in thecameras, the field of view of the lens, the orientation of the cameras,the positions of the cameras, the color spectrum of the cameras, theframe rate of the cameras, the exposure time of the cameras, the gain ofthe cameras, the aperture size of the lenses, or the like. An imageprocessing apparatus is then used to process the images from the atleast two different cameras to provide optimal stereoscopic vision to adisplay.

In a preferred embodiment according to the present invention, thedifference between the at least two cameras is the distortion profile ofthe wide-angle lenses used or the resulting modified distortion profileof the camera after smart-binning by the sensor or the cameraprocessing. One such example, when the lenses have different distortionprofile, is when one of the wide-angle lens has a distortion profilewith enhanced resolution in the central region of the field of viewwhile the other wide-angle lens has a distortion profile with enhancedresolution toward the edges of the field of view. The images from thesetwo cameras are then combined inside a processing unit. The final resultis two images having a resolution in the whole field of view higher thanthe original resolution of each original image while keeping thegeometrical differences due to parallax to create dual displays for ahuman interpreted by the brain as 3D vision. Another example, when thecameras themselves output images with different distortion profilesinstead of due to differences in the lenses, is when the distortion ofthe image is modified either by smart-binning done by the sensor or byprocessing inside the camera that modify the distortion of the imagebefore output. This type of distortion by the sensor or the camera canalso be dynamics, changing in time according to the movement of objectsin the field of view, the direction of gaze of the user, or the like.

In another embodiment of the present invention, the difference betweenthe at least two cameras is the orientation of the optical axis which isoffset between each other, meaning there is an angle between the camerasoptical axis. This angle can be a large angle set voluntary or a smallinvoluntary alignment error between the cameras. In this exampleembodiment, because of the tilt angle between the cameras, only aportion of the total field of view of each wide-angle lenses is used toimage in double the scene for stereographic display and a part of thefield of view is only visible to each camera. The images from these atleast two cameras are then combined inside a processing unit. Since theprocessing unit knows the distortion profile of the wide-angle lensesand the difference of orientation between the cameras, the processingalgorithm can create a full view of the scene for both eyes. The resultis an enlarged total field of view of the system where only a part ofthe scene, sometime a desired region of interest, is imaged by bothcameras and displayed in three dimensions.

In another embodiment of the present invention, the difference betweenthe at least two cameras is the field of view of each lens, one beingwider than the other. In this example embodiment, only a portion of thewider field of view imaged by the wider field of view camera is alsoimaged by the narrower field of view camera. The images from these twocameras are then combined inside a processing unit. Since the processingunit knows the field of view and distortion profile of each lens, theprocessing algorithm can create a full view of the scene for both eyes.In the part of the field of view imaged by both the wider and thenarrower cameras, the processing algorithm display different views foreach eye due to parallax difference from the multiple capturing positionwhile in the part of the field of view seen by only the wider camera,the two generated views for the display are identical without anyparallax difference. In some embodiment of the present invention, theresolution in pixels per degree in the narrower field of view camera ishigher than in the wider field of view camera and more details can beidentified from the narrower field of view camera. The processingalgorithm then use the higher resolution from the narrower camera aswell as the geometrical difference between the two resulting images dueto the parallax difference from different capture point to create twoviews of higher resolution while keeping the geometrical differences dueto parallax to generate 3D display.

In another embodiment of the present invention, the difference betweenthe at least two cameras is the light spectrum of the cameras. One suchexample is when combining together a visible light camera to aninfra-red light camera. The images from these two cameras are thencombined inside a processing unit. Since the processing unit knows thefield of view and distortion profile of each lens, the processingalgorithm can create displays with a full view of the scene for botheyes. The geometrical differences due to the parallax from the twocamera difference of capturing position can be calculated by theprocessing algorithm and depending on the application, the processedimages using the textures from either the visible camera or theinfra-red camera are displayed.

In another embodiment of the present invention, the difference betweenthe at least two cameras is the frame rate. In this example embodiment,one camera could be a camera capturing a higher number of frames persecond and the other a camera capturing a lower number of frames persecond, including the limit case of using only a still image. Theprocessing algorithm can then use the information from the higher framerate camera to create the two required display for stereoscopic visionwith a high frame rate and use the images from the camera having a lowernumber of frames per second to adjust the geometrical differences due toparallax and improve the display. This adjustment of 3D is limited bythe lower frame rate camera and is done less often than at each frame ofthe higher frame rate camera.

In another embodiment of the present invention, the difference betweenthe at least two cameras is either the exposure time, the gain or theaperture size (f/#). By having a different exposure time, gain oraperture size, the at least two cameras can see in a larger dynamicrange. In one of the two resulting images, from the camera having alonger exposure time, a larger gain or a larger aperture (lower f/#),brighter objects might be over exposed while other darker objects wouldbe perfectly exposed in this image. In the other image from the othercamera, brighter objects would be perfectly exposed while darker objectswould be under exposed. Even if some part of the images are over orunder exposed, the geometrical differences due to a difference ofcapture position would still be visible to the processing algorithm. Theprocessing algorithm can then produce two views for stereoscopic displayusing the whole high dynamic range captured from the multiple cameraswhile still keeping the parallax difference in the images.

In another embodiment of the present invention, the optical distortionof the two lenses in the two cameras are configured so that theoutputted images are already pre-distorted in exactly the same distortedway required for the display unit, for example in an augmented realitydevice or a see-through device. This allow to display the images fromthe cameras to a user without any lag or delay associated to imageprocessing to create the required distorted images compatible with thedisplay. In this embodiment, each camera can be different to account forthe difference between the left and the right eye of the observer thatwould be otherwise processed in a usual display without pre-distortionlenses. One example embodiment of the present invention is a see-throughdevice made from fixing a mobile phone. On this mobile phone, the twocameras are placed on the back of the device and the front of the devicehas a display. When using the mobile phone inside a cardboard virtualreality headset or the like, the result can be an augmented realitypresenting the content from each camera to each eye without furtherimage distortion processing inside the phone.

In a last embodiment of the present invention, the cameras used forstereoscopic vision could combine multiple of the above difference ofparameters. For example, not in any way limiting the possiblecombinations of the above embodiments, two user could use their mobiledevice each having a camera looking at a scene with some overlap. Thecameras could have different distortion profile, field of view,orientation, exposure setting, frame rate and spectrum all at the sametime. By providing all the information about each camera to theprocessing algorithm, it can then properly detect which zone overlap andcreate two optimal views to be displayed to a user and see 3D in onlythe part of the field of view imaged by multiples cameras.

In all of the above embodiments, the processing algorithm receives andprocess the image from the at least two cameras having differentparameters. Since the processing algorithm knows the exact parameters ofthe cameras (field of view, resolution, distortion, orientation, colorspectrum, etc), the processing algorithm can reconstruct dual 2D viewsgenerated exactly for a display specific to each eye in an stereoscopicdisplay system while using the optimal information from each camera. Insome embodiment, while reprocessing the distortion to create 2D views,the processing algorithm can correct small alignment error (unwantedtilt) of the camera by modifying distortion of the displayed images andcan be used to enhance the calibration between stereoscopic cameras.When viewed by a human, the brain then interpret these dual 2D views asa normal vision of a 3D scene.

The processing algorithm can also adjust the 2D views generated for eacheye to account for movement of the stereoscopic display with respect toa central initial point. When the display is in an initial centralposition, the amount of parallax visible in the objects seen by the atleast two cameras is due to the distance from the two capture positions.When the display move, for example when the head of an user for avirtual reality headset move up, down, left, right, forward or backward,the processing algorithm can adjust the distortion of the generateddisplay to compensate for the head movements, giving the illusion ofmoving inside the displayed images even if the cameras that captures theoriginal images are at fixed positions.

In all of the above embodiments, the at least two different cameras aswell as the processing algorithm can be on the same device or ondifferent devices. Some examples of devices that can be equipped witheither these cameras, processing algorithm or both include, but in noway limited to, a smartphone, a standalone camera, a virtual realitydisplay device, an augmented reality display device or the like.

In all of the above embodiments, in addition to using the at least twocameras to capture the scene with parallax information used to calculate3D information about the scene and create an apparent 3D view bygenerating a different 2D view for each display, the processingalgorithm can further enhance the 3D information of the scene by usinginformation from any source.

In stereoscopic vision systems, the positions of the cameras allow tochange the perception of user observing the display. For example, whenthe cameras are positioned at a low height compared to his eyes, theuser looking at the stereoscopic display will have the feeling of beingshorter than he is. Alternatively, when the cameras are positioned abovethe height of his eyes, looking at the stereoscopic display will createthe feeling of being taller. In some embodiments of the presentinvention, by using pairs of cameras at various heights on a deviceallows the final user to choose the desired point of view, short ortall. This can be used to better understand the point of view of someoneelse like a small kid, a person sitting in a wheelchair, or a very tallperson. Combined with the processing algorithm according to the presentinvention, the display can smoothly switch from a display to the other,including positions between the cameras using a processed displayposition.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The foregoing summary, as well as the following detailed description ofa preferred embodiment of the invention, will be better understood whenread in conjunction with the appended drawings. For the purpose ofillustration, there is shown in the drawings an embodiment which ispresently preferred. It should be understood, however, that theinvention is not limited to the precise arrangements andinstrumentalities shown.

In the drawings:

FIG. 1 is an embodiment of the present invention where the differencebetween the cameras is the distortion profile of the lenses;

FIG. 2 is an embodiment of the present invention where the differencebetween the cameras is the orientation of the optical axis of thelenses;

FIG. 3 is an embodiment of the present invention where the differencebetween the cameras is the field of view of the lenses;

FIG. 4 is an embodiment of the present invention where the differencebetween the cameras is the color spectrum;

FIG. 5 is an embodiment of the present invention where the differencebetween the cameras is the frame rate of the cameras;

FIG. 6 is an embodiment of the present invention where the differencebetween the cameras is the exposure, the gain and/or the aperture sizeof the lenses;

FIG. 7 is an embodiment of the present invention where the differencebetween the cameras is the distortion profile of the lenses, with bothdistortion profile designed exactly to reduce processing for a displaywithout delay;

FIG. 8 is an embodiment of the present invention where multiplesdifferences between the cameras are combined; and

FIG. 9 is an embodiment of the present invention where the differencebetween the cameras is the height of the capture.

DETAILED DESCRIPTION OF THE INVENTION

The words “a” and “an”, as used in the claims and in the correspondingportions of the specification, mean “at least one.”

FIG. 1 shows an embodiment according to the present invention where thedifference between the at least two cameras is the distortion profile ofthe lenses. In some embodiments of the present invention, the differenceof distortion profile between the at least two cameras is such that theoutput resolution from the cameras has a difference of at least 10% inangular resolution for at least one angular field. This angular field isrepresented by an object far away in the scene such that it is imaged byboth cameras at the same angular position relative to their opticalaxis. The at least 10% angular resolution difference can be measured inpixels/degree in the output image, in micrometers/degree in the imageplane or by any other similar unit of measurement where a ratio of adistance on the image plane by a unit of an angular displacement in theobject scene relative to an optical axis is used. This method formeasuring the difference of distortion between the two cameras is justan example and other methods can be used to measure an intentionaldifference of the distortion of the lenses or the cameras according tothe present invention.

A scene 100 comprises of multiples objects 102, 104 and 106 to be imagedby at least two cameras. In this example, both the cameras have awide-angle field of view, but this is not a requirement according to thepresent invention. The camera 112 with lens 110 has a distortion profile111 with increased magnification in the center of the field of view andlower magnification toward the edges, creating the image 120. The imageof the human person 104 is in the center and hence with high resolutionor bigger, while the image of the tree 102 and of the sun 106 are inlower resolution, or smaller. The camera 117 with lens 115 has adistortion profile 116 with increased magnification toward the edges ofthe field of view and lower magnification in the center, creating theimage 125. The image of the human person 104 is in the center and hencewith lower resolution, while the image of the tree 102 and of the sun106 are in higher resolution. The images 120 and 125 from the twocameras 112, 117 are then stored or transmitted at 130 to be used now orlater by the processing unit 140. This transmission can be internallyinside a device integrating the cameras, the processing unit and thedisplay or it can be across multiples devices via a communication link,including a connection by a wire or over the Internet. The processingunit 140 can be a hardware or a software implementation having thealgorithm to combine the two images. The distortion profile 111, 116 ofthe two lenses 110, 115 are known to the processing unit either becauseit was transmitted with the images via a marker or a metadata or becausethe processing unit was pre-configured with the distortion profiles 111,116 of the lenses 110, 115. In addition to information from the cameras112, 117, the processing unit 140 can also receive any other externalinformation to improve the processing of the images, includinginformation from a database, from a user or from an artificialintelligence algorithm having processed past images via deep learningtechniques or other artificial intelligence learning techniques. Sincethe distortion profile 111, 116 of the two lenses 110, 115 are perfectlyknown to the processing unit 140, the processing algorithm can createdewarped views for each eye removing all the distortion from each lenses110, 115 or modifying the distortion as required. The resultingdifference in geometry in the dewarped views are due to parallaxdifference between the two cameras 112, 117 capturing the scene fromdifferent locations and can be used to create the depth perception inthe stereographic view. The processing algorithm then further enhancesthe central resolution of the view coming from the lens having anenhanced resolution toward the edge by using the information from theother camera having enhanced resolution toward the center. The same isdone for the other view. The final result from the processing unit 140is two images having a resolution in the whole field of view higher thanthe original resolution of each original image while keeping thegeometrical differences due to parallax. The two images are thentransferred to a display unit 150 that present to a human observer thetwo stereoscopic views with enhanced resolution compared to theoriginally captured images. In another embodiment of the presentinvention, instead of the lens 110 and 115 having a different distortion111, 116, the images with different distortion 120 and 125 can beoutputted from the cameras themselves. The different distortion in theimages 120 and 125 is then resulting from processing inside the cameraswhere a higher resolution image is compressed on the side at image 120and in the center at image 125. This can be done by either software orhardware processing of the original images received by the camera of bysmart-binning by the sensor where the sensor down-sample the resolutionin a part of the image by combining multiples pixels together. Then, aswith the case where the difference of distortion is produced by thelenses, the output images are stored or transmitted at 130 to be usednot or later by the processing unit 140 until displayed at 150. Thistype of distortion 113, 118 modified inside the cameras 112, 117 bysensor smart-binning, hardware or software processing or by an activeoptical mean can also be dynamics, changing the distortion in timeaccording to the movement of objects in the field of view, the directionof gaze of the user, or the like.

In some embodiments of the present invention, the resulting resolutionof the two displayed images are not equal, with a higher resolutionimage displayed to the eye of the user having ocular dominance. Thedominant eye is the eye from which visual input are preferred from theother eye by the brain.

FIG. 2 shows an embodiment according to the present invention where thedifference between the at least two cameras is the orientation of theoptical axis of the lenses inside the cameras. A scene 200 comprises ofmultiples objects 202, 204 and 206 in a scene to be imaged at leastpartially by at least two cameras. In this example figure, the lens 210is tilted intentionally or not toward the left of the image while thelens 215 is tilted intentionally or not toward the right of the image.In other embodiments, the tilt angle between the 2 cameras could also benegative instead of positive, with the cameras facing inward instead ofoutward. The resulting image 220 from lens 210 can image the tree 202and the human 204, but cannot see the sun 206. The resulting image 225from lens 315 can image the human 204 and the sun 206, but not the tree202. The images 220 and 225 from the two cameras are then stored ortransmitted at 230 to be used now or later by the processing unit 240.The processing unit 240 can be a hardware or a software implementationhaving the algorithm to combine the two images. The exact orientation ofthe two lenses are known to the processing unit either because it wastransmitted with the images via a marker or a metadata or because theprocessing unit was pre-configured with the orientation of the lenses.In the part of the field of view imaged by both lenses, as the human 204in this example, the processing algorithm 240 creates different viewsfor each eye due to parallax difference from the multiple capturingposition. In the part of the field of view seen by only one camera, asthe tree 202 or the sun 206 in this example, the generated views for thedisplay are identical without any parallax difference. The final resultis two views transmitted to the display device 250 that are either in 2Dor 3D depending on the direction the user looks at. The transitionbetween the 2D and 3D viewing area is minimized via a blend to avoiddiscomfort to the human observer.

In some embodiments of the present invention, the missing 3D informationin the part of the scene image by only a single lens can be obtained viaan additional source. The processing unit can then use this additionalinformation to further reconstruct the 3D scene and extend the part ofthe scene viewed in 3D.

FIG. 3 shows an embodiment according to the present invention where thedifference between the at least two cameras is the field of view of thelenses. A scene 300 comprises of multiples objects 302, 304 and 306 tobe imaged fully by the wide-angle lens 310 and partially by thenarrow-angle lens 315. The resulting image from lens 310 is image 320where the tree 302, the human 204 and the sun 306 are all visible. Theresulting image from the lens 315 is image 325 where only the human 304is visible. Because the lens 310 is wide-angle, the average resolutionin pixels/degree for imaged objects is generally lower than with thenarrow-angle lens 315. For the image of the human 304, in addition tothe geometrical difference between the images due to parallax from thedifferent capturing positions, the resolution is higher in image 325than 320. The images 320 and 325 from the two cameras are then stored ortransmitted at 330 to be used now or later by the processing unit 340.The processing unit 340 can be a hardware or a software implementationhaving the algorithm to combine the two images. The exact field of viewof the two lenses are known to the processing unit either because it wastransmitted with the images via a marker or a metadata or because theprocessing unit was pre-configured with the field of view of the lenses.In the part of the field of view imaged by both lenses, as the human 304in this example, the processing algorithm 340 creates different viewsfor each eye due to parallax difference from the multiple capturingposition. Since the resolution is generally different between the twoimages, the textures from the highest resolution image available areused to generate the two views in higher resolution in the part of thefield of view images by multiples cameras. In the part of the field ofview imaged only by the wide-angle lens, as in the tree 302 and the sun306, both views generated are identical and are generated from the image320. The two generated views are then transmitted to the display unit350. The transition between the 2D and 3D viewing area and higher tolower resolution viewing area is minimized via a progressive blend alongthe images to avoid discomfort to the human observer. In otherembodiments, the 3D in the part of the field of view imaged only by thewide-angle lens can be can be generated by A.I. processes analyzing thescene, software or hardware processes or manual adjustment. In thiscase, even outside of the narrow-angle field of view, the two generatedviews for display are different using this 3D information.

FIG. 4 shows an embodiment according to the present invention where thedifference between the at least two cameras is the color spectrum of thelenses. A scene 400 comprises of multiples objects 402, 404 and 406 tobe imaged fully in the visible spectrum by the wide-angle lens 410 andimaged fully in the infra-red spectrum by the wide-angle lens 415. Inthis example, the pictures are taken in low-light conditions and theimage 420 resulting from the visible camera can barely identify thehuman 404 because of the low light. However, the human 404 is at ahigher temperature than its surrounding and emit a lot of infra-redlight. In the image 425 from the infra-red lens 415, the human 404 iseasily visible. The images 420 and 425 from the two cameras are thenstored or transmitted at 430 to be used now or later by the processingunit 440. The processing unit 440 can be a hardware or a softwareimplementation having the algorithm to combine the two images. The colorspectrum of the two lenses are known to the processing unit eitherbecause the information was transmitted with the images via a marker ora metadata or because the processing unit was pre-configured with thecolor spectrum of the lenses. When creating the two views forstereoscopic display, the processing algorithm 440 creates differentviews for each eye due to parallax difference from the multiplecapturing position. When an object is clearly more visible in one of thetwo images, as the human 404 clearly more visible in image 425 than inimage 420, the processing unit display the same content on bothgenerated displays for 2D view. When an object is visible in both thevisible and infra-red spectrum as the moon 406, the processing unitcombine the geometrical difference between the objects to create adifference of parallax in the generated views. The textures to bedisplayed in the final output are either from the visible or theinfra-red lens depending on the application. The two generated views arethen transmitted to the display unit 450.

FIG. 5 shows an embodiment according to the present invention where thedifference between the at least two cameras is the frame rate of thecameras. A scene 500 comprises of multiples objects 502, 504 and 506 tobe imaged by at least two cameras. In this example, both the camerashave a wide-angle field of view. The camera with lens 510 can captureimages at a lower frame rate, creating the images 520. The camera withlens 515 can capture images at a higher frame rate, creating the images525. The images 520 and 525 from the two cameras are then stored ortransmitted at 530 to be used now or later by the processing unit 540.Since the number of frame in 520 is lower than the number of frame 525,the processing unit use mainly the images 525 to generate at a highframe rate the two images to display. When the processing unit receive anew image 520 from the lower frame rate camera, it can update theparallax information for the next few generated images until a new imageis received from the lower frame rate camera. When a new image 520 isreceived, the parallax information are again updated. In someapplications, the lower frame rate camera can be a camera providing onlya single static frame 520 and the processing algorithm use it only tocalculate the geometrical differences between the single image 520 andall the high frame rate images 525. In another embodiment, the lowerspeed camera could be activated only when movement is detected in theimage from the higher speed camera. The inverse could also be done, withthe higher speed camera activated or the frame rate increased only whenmovement is detected in the lower frame rate camera. The two generatedviews are then transmitted to the display unit 550.

FIG. 6 shows an embodiment of the present invention where the differencebetween the cameras is the exposure, the gain and/or the aperture sizeof the lenses. By having a different exposure time, gain or aperturesize, the at least two cameras can see in a larger dynamic range. Ascene 600 comprises of multiples objects 602, 604 and 606 to be imagedby at least two cameras. The camera 610 having a longer exposure time, alarger gain or a larger aperture (lower f/#) creates image 620. In image620, brighter objects as the human 604 might be over exposed while otherdarker objects like the tree 602 and 606 would be perfectly exposed inthis image. The camera 615 having a shorter exposure time, a smallergain or a smaller aperture (higher f/#) creates image 625. In image 625,brighter objects as the human 604 would be perfectly exposed whiledarker objects like the tree 602 and 606 would be underexposed. Theimages 620 and 625 from the two cameras are then stored or transmittedat 630 to be used now or later by the processing unit 640. Even if somepart of the images are over or under exposed, the geometricaldifferences due to a difference of capture position would still bevisible to the processing algorithm and it can create the correspondingparallax difference in the images. For the texture, the processingalgorithm uses the part of the images 620 or 625 with perfect exposureto generate the two display, creating an output with a higher dynamicrange than the two original cameras themselves. The two generated viewsare then transmitted to the display unit 650, having an higher dynamicrange (HDR) than each individual original images.

FIG. 7 shows an embodiment of the present invention where the differencebetween the cameras is the distortion profile of the lenses, with bothdistortion profile designed exactly to reduce processing for a displaywithout delay. A scene 700 comprises of multiples objects 702, 704 and706 to be imaged by at least two cameras. In one example according tothe present embodiment, in no way limiting the scope of the invention,two cameras 712 and 714 are located on the back of a mobile phone device710. The lens 712 is designed to output directly the image 720 with thedistortion and field of view matching the requirement of the left eye atdisplay 750 and 755. The lens 714 is designed to output directly theimage 725 with the distortion and field of view matching the requirementof the right eye at display 750 and 755. This way, the distortionprofiles allows to minimize or to avoid completely the distortionprocessing before they are displayed. The images 720 and 725 from thetwo cameras are then stored or transmitted at 730 to be used now orlater. Since the distortion of the output from the lens is alreadypre-distorted to match the requirements of the display 750 and 755, thetransmission can be directly from storage 730 to display 750 or 755without using the processing unit 740. In other embodiments, the lenshaving distortion matching the requirement of the display can becombined to any other difference of parameter described before and inthis case the optional processing unit 740 can be used for optimaldisplay even with the difference of parameter. The front of the mobilephone device 710 can be used as the display when the phone is insertedinside a cardboard viewer to create a real-time augmented reality system750 with see-through capabilities or a playback virtual reality system755. In another embodiment, the difference between the cameras is thedistortion profile outputted from the cameras instead of the distortionprofile of the lenses. The different distortion in the images 720 and725 is then resulting from processing inside the cameras to create thedesired pre-distorted images. This can be done by either software orhardware processing of the original images received by the camera of bysmart-binning by the sensor where the sensor down-sample the resolutionin a part of the image by combining multiples pixels together.

FIG. 8 shows an embodiment where several difference of parameters arecombined according to the present invention. A scene 800 comprises ofmultiples objects 802, 804 and 806 to be imaged by at least two cameras812 and 816 located on two different devices, respectively 810 and 814.The invention is not limited to two cameras and additional cameras canbe used as the mobile phone 818 having camera 819. The camera 812produces the image 820 having a wide-angle field of view, its uniquedistortion profile, a normal exposure, a high resolution image and acentral orientation. The camera 816 produces image 825 with a narrowfield of view, its unique distortion profile, a lower exposure, a highresolution image and a central orientation. The optional camera 819produces image 827 with a narrow field of view, its unique distortionprofile, a normal exposure, a lower resolution image and a tiltedorientation toward the right. The images 820, 825 and other optionalimages 827 from the at least two cameras are then stored or transmittedat 830 to be used now or later by the processing unit 840 that generatetwo optimal views and then transmits them to the display unit 850.

FIG. 9 shows an embodiment of the present invention where the differencebetween the cameras is the height of the capture to represent variouscases of seeing vision through the eyes of someone else. Thestereoscopic images are captured by a capture device 905, 915 or 925 atthe height of the eyes of various people. In this example figure, in noway limiting the scope of this invention, the people capturing areeither a tall adult 900, a sitting person or someone in a wheel chair910 or kid or a short person 920. The images from the capture devices905, 915 or 925 are then stored or transmitted at 930 to be used now orlater by the processing unit 940. The final observer 950 looking at thedisplay through a virtual reality device can then see the point of viewof any of the people 900, 910 or 920 as desired.

In some embodiments according to the present invention, instead ofgenerating two output images for display to a human using a head-mountedvirtual reality headset, an augmented reality headset or a mobile deviceinserted in a headset, the processing unit uses the images from thestereoscopic vision system to analyze the scene and output the resultinganalysis to an algorithm unit. This algorithm unit can be any unitcapable of analyzing the images, including, but not limited to, asoftware algorithm, a hardware algorithm or an artificial intelligenceunit based or not on a neural network and trained or not via deeplearning techniques or the like. The algorithm unit can thenautomatically use the information extracted from the at least twodifferent images and processed by the processing unit for anyapplication it requires, including for generating distance informationabout a scene including information about distance from a origin point,to generate higher quality image with enhanced image quality usinginformation extracted from the algorithm unit, to generate informationused in an artificial intelligence algorithm including artificialintelligence algorithm trained via deep learning neural networks or thelike or to generate a single image with superposed left eye and righteye images to be separated via active or passive glasses, either colorfilter, polarized glasses, synchronized shutter glasses or the like.

All of the above are figures and examples of specific image distortiontransformation units and methods. In all these examples, the imager canhave any field of view, from very narrow to extremely wide-angle. Theseexamples are not intended to be an exhaustive list or to limit the scopeand spirit of the present invention. It will be appreciated by thoseskilled in the art that changes could be made to the embodimentsdescribed above without departing from the broad inventive conceptthereof. It is understood, therefore, that this invention is not limitedto the particular embodiments disclosed, but it is intended to covermodifications within the spirit and scope of the present invention asdefined by the appended claims.

What we claim:
 1. An image acquisition system for capturing a scene, thesystem comprising: a. a first camera having a plurality of first imagingparameters and a first capture position relative to the scene, the firstcamera being configured to capture a first output image of the scene; b.a second camera having a plurality of second imaging parameters and asecond capture position relative to the scene, the second camera beingconfigured to capture a second output image of the scene, the first andsecond capture positions being different from each other, one or more ofthe first imaging parameters being different from a corresponding one ormore of the second imaging parameters, the first and second outputimages being different from each other according to the differing firstand second capture positions and the one or more differing first andsecond imaging parameters; c. a processing unit connected to the firstand second cameras, the processing unit being configured to: i. receivethe first and second output images from respective first and secondcameras, and ii. process the first and second output images according toa geometrical difference due to parallax from the first and secondcapture positions and according to any remaining differences due to theone or more differing first and second imaging parameters, in order toproduce first and second processed images, wherein the one or morediffering first and second imaging parameters includes at least adifference in lens distortion profiles between the first and secondcameras or a difference in camera distortion profiles between the firstand second cameras, wherein the first camera has a camera distortionprofile or a first lens of the first camera has a lens distortionprofile with increased magnification in a first zone of a field of viewand the second camera has a camera distortion profile or a second lensof the second camera has a lens distortion profile with increasedmagnification in a second zone of the field of view, the second zonebeing different than the first zone, and wherein, to create at least onecombined image, the processing unit is further configured to at leastone of: (1) combine information from the first output image outside ofthe first zone of the field of view with information having increasedmagnification from the second camera, or (2) combine information fromthe second output image outside of the second zone of the field of viewwith information having increased magnification from the first camera,wherein the processing unit is further configured to: iii. pre-storedifference information regarding the difference in the field of view ofeach of the first and second cameras, iv. receive manual input of thedifference information from a user, or v. receive the differenceinformation from the first and second cameras written in a marker and/ormetadata.
 2. The system of claim 1, further comprising at least onedisplay for displaying the first and second processed images.
 3. Thesystem of claim 2 wherein the at least one display is on one of ahead-mounted virtual reality headset, an augmented reality headset, or amobile device capable of insertion into a headset.
 4. The system ofclaim 1 wherein the first and second capture positions are modifiable tochange the desired view of the scene.
 5. The system of claim 1, whereinthe at least one combined image has enhanced image resolution.
 6. Thesystem of claim 1, wherein the at least one combined image includes 3Dinformation.
 7. An image acquisition system for capturing a scene, thesystem comprising: a. a first camera including one or more lensescreating a first distortion profile, the first camera having a firstcapture position relative to the scene and being configured to capture afirst output image of the scene; b. a second camera including one ormore lenses creating a second distortion profile different from thefirst distortion profile, the second camera having a second captureposition relative to the scene and being configured to capture a secondoutput image of the scene, the first and second capture positions beingdifferent from each other, the first and second output images beingdifferent from each other according to the differing first and secondcapture positions and the differing first and second distortionprofiles; and c. a processing unit configured to create at least onecombined image by at least one of: i. combining information from thefirst output image outside of a first zone of a field of view withinformation having increased magnification from the second camera, orii. combining information from the second output image outside of asecond zone of the field of view different from the first zone withinformation having increased magnification from the first camera,wherein the first and second distortion profiles respectively matchrequirements of a left eye and a right eye of a user at a display andare configured to minimize or avoid completely the processing of thedistortion in the first and second images before they are displayed tothe user, wherein difference information regarding the difference in thefirst and second lens distortion profiles of the first and secondcameras is pre-stored, received from a user, or received from the firstand second cameras written in a marker and/or metadata.
 8. The system ofclaim 7, further comprising at least one display configured to displaythe first and second output images.
 9. The system of claim 8 wherein theat least one display is on at least one of a head-mounted virtualreality headset, an augmented reality headset, or a mobile devicecapable of insertion into a headset.
 10. The system of claim 7 whereinthe first and second capture positions are modifiable to change thedesired view of the scene.
 11. An image acquisition system for capturinga scene, the system comprising: a. a first camera creating a firstdistortion profile either via smart-binning by a sensor or viaprocessing inside the camera, the first camera having a first captureposition relative to the scene and being configured to capture a firstoutput image of the scene; b. a second camera creating a seconddistortion profile either via smart-binning by a sensor or viaprocessing inside the camera, the second distortion profile beingdifferent from the first distortion profile, the second camera having asecond capture position relative to the scene and being configured tocapture a second output image of the scene, the first and second capturepositions being different from each other, the first and second outputimages being different from each other according to the differing firstand second capture positions and the differing first and seconddistortion profiles; and c. a processing unit configured to create atleast one combined image by at least one of: i. combining informationfrom the first output image outside of a first zone of a field of viewwith information having increased magnification from the second camera,or ii. combining information from the second output image outside of asecond zone of the field of view different from the first zone withinformation having increased magnification from the first camera,wherein the first and second distortion profiles respectively matchrequirements of a left eye and a right eye of a user at a display andare configured to minimize or avoid completely the processing of thedistortion in the first and second images before they are displayed tothe user, wherein the processing unit is further configured to pre-storedifference information regarding the difference in the first and secondcamera distortion profiles of the first and second cameras, receive thedifference information from a user, or receive the differenceinformation from the first and second cameras written in a marker and/ormetadata.
 12. The system of claim 11, further comprising at least onedisplay configured to display the first and second output images. 13.The system of claim 12 wherein the at least one display is on at leastone of a head-mounted virtual reality headset, an augmented realityheadset, or a mobile device capable of insertion into a headset.
 14. Thesystem of claim 11 wherein the first and second capture positions aremodifiable to change the desired view of the scene.
 15. An imageacquisition system for analyzing information about a scene, the systemcomprising: a. a first camera having a plurality of first imagingparameters and a first capture position relative to the scene, the firstcamera being configured to capture a first output image of the scene; b.a second camera having a plurality of second imaging parameters and asecond capture position relative to the scene, the second camera beingconfigured to capture a second output image of the scene, the first andsecond capture positions being different from each other, one or more ofthe first imaging parameters being different from a corresponding one ormore of the second imaging parameters, the first and second outputimages being different from each other according to the differing firstand second capture positions and the one or more differing first andsecond imaging parameters; c. a processing unit connected to the firstand second cameras, the processing unit being configured to: i. receivethe first and second output images from the respective first and secondcameras, and ii. process the first and second output images according toa geometrical difference due to parallax from the first and secondcapture positions and according to any remaining differences due to theone or more differing first and second imaging parameters, in order toanalyze the scene, wherein the one or more differing first and secondimaging parameters includes at least a difference in lens distortionprofiles between the first and second cameras or a difference in cameradistortion profiles between the first and second cameras, wherein thefirst camera has a camera distortion profile or a first lens of thefirst camera has a lens distortion profile with increased magnificationin a first zone of a field of view and the second camera has a cameradistortion profile or a second lens of the second camera has a lensdistortion profile with increased magnification in a second zone of thefield of view, the second zone being different than the first zone, andwherein, to create at least one combined image, the processing unit isfurther configured to at least one of: (1) combine information from thefirst output image outside of the first zone of the field of view withinformation having increased magnification from the second camera, or(2) combine information from the second output image outside of thesecond zone of the field of view with information having increasedmagnification from the first camera, wherein the processing unit isfurther configured to: iii. pre-store difference information regardingthe difference in the field of view of each of the first and secondcameras, iv. receive manual input of the difference information from auser, or v. receive the difference information from the first and secondcameras written in a marker and/or metadata.
 16. The system of claim 15,wherein the at least one combined image has enhanced image resolution.17. The system of claim 15, wherein the at least one combined imageincludes 3D information.